Skip to content

Passed Predict Params to RAPS for conformity score calculation and EnsembleClassifier #784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Passed Predict Params to RAPS for conformity score calculation and EnsembleClassifier #784

wants to merge 3 commits into from
+48 −4

Conversation

@VishnoiAman777
Copy link

@VishnoiAman777 VishnoiAman777 commented Oct 30, 2025 •
edited
Loading

Description

This change adds a test case to ensure that predict_params are correctly passed and handled when using Ensemble Classifier to predict_proba function for an estimator and RAPSConformityScore Class. Addtional test test_raps_with_predict_params is added to verify that the predict_params are properly passed through the fit and predict methods in RAPSConformityScore.

Fixes #614

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

The following test was added to verify the changes:

  • test_raps_with_predict_params
    • This test verifies that predict_params are correctly passed through the fit and predict methods when using RAPSConformityScore

Checklist

  • I have read the contributing guidelines
  • I have updated the HISTORY.rst and AUTHORS.rst files
  • Linting passes successfully: make lint
  • Typing passes successfully: make type-check
  • Unit tests pass successfully: make tests
  • Coverage is 100%: make coverage
  • When updating documentation: doc builds successfully and without warnings: make doc
  • When updating documentation: code examples in doc run successfully: make doctest

@allglc
Copy link
Collaborator

allglc commented Oct 30, 2025

Hello @VishnoiAman777, thank you for the contribution!

It seems that indeed predict_params were not passed to the estimator in predict_proba_calib for the "prefit" case. Good catch!

predict_params were also not passed to the estimator in RAPSConformityScore. Do you think it is the only score where the issue happens? I think it is but I would like your opinion.

I have made comments on some of your modifications, please look at them when you have the time.

Otherwise, I was wondering about the possible uses of predict_params. Is it only used by gradient boosting models? Are you aware of another use?

@VishnoiAman777
Copy link
Author

VishnoiAman777 commented Oct 31, 2025

Hi @allglc,
Thank you for your response.

The RAPSConformityScore is the only case where we require predict_params. This is because of k_star and lambda required in RAPS. As far as I can tell from reviewing the codebase, this requirement does not apply elsewhere.

As you have said predict_params is typically needed in Boosting implementations, many third-party libraries—such as XGBoost and CatBoost—also support them. My rationale for including them is to ensure compatibility with any estimator that follows the scikit-learn convention. This also allows for their use in custom estimators and within scikit-learn pipeline objects.

Additionally, I am not able to see any comments on my changes yet as you have mentioned. Have you started a code review?

allglc
allglc requested changes Nov 3, 2025
View reviewed changes
mapie/conformity_scores/sets/raps.py
self.y_pred_proba_raps = self.predictor.single_estimator_.predict_proba(
self.X_raps
)
predict_params = kwargs.pop("predict_params", None)
Copy link
Collaborator

@allglc allglc Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you only need to add **predict_params to the function? i.e,:
self.y_pred_proba_raps = self.predictor.single_estimator_.predict_proba(self.X_raps, **predict_params) like you did in the classifier.py file.

mapie/tests/test_classification.py
np.testing.assert_equal(y_pred, 0)


def test_raps_with_predict_params() -> None:
Copy link
Collaborator

@allglc allglc Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test does not seem to test what we want. It does not test if predict params are passed into RAPS and actually this test passes on the current version of MAPIE (i.e., without your modifications).

Furthermore, the code is too complex: there is no need to recreate a dataset. Please refer to e.g., test_predict_parameters_passing for inspiration.

@allglc
Copy link
Collaborator

allglc commented Nov 3, 2025

@VishnoiAman777 Thanks for the clarification!

Sorry I forgot submitting the review, you should see the comments now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@allglc allglc allglc requested changes

@LacombeLouis LacombeLouis Awaiting requested review from LacombeLouis

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

predict_params is not passed everywhere in MapieClassifier in some settings

2 participants

@VishnoiAman777 @allglc